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Abstract 

Text-based  passwords  are  not  necessarily  as  secure  as  they  appear  because  the 
protocols  required  for  a  secure  password  are  typically  ignored  for  a  more  user- 
friendly  one.  In  this  paper,  we  propose  a  new  sketch-based  password  system 
that  uses  biometric  data  and  provides  exceptional  tradeoff  between  security  and 
usability.  By  extending  a  shape  recognition  algorithm  called  Simple  K-Space 
(SKS),  a  system  that  is  capable  of  being  both  secure  and  robust  is  developed.  In 
theory  and  experimentation,  new  insights  into  the  SKS  philosophy  are  provided, 
including  uniqueness  of  the  model  and  tolerance  of  the  framework.  This  system 
is  compared  with  a  state-of-the-art  graphical  password  system  using  Dynamic 
Time  Warping  (DTW)  and  found  to  achieve  comparable  performance  without 
using  any  biometrics.  The  addition  of  biometrics  substantially  increased  the 
level  of  performance,  reducing  the  equal  error  rate  (EER)  by  more  than  12%. 
Keywords:  Security,  Authentication,  Password,  Biometrics,  Drawmetrics, 
Sketches 


1.  Introduction 

In  most  secured  systems,  text-based  passwords  are  the  golden  standard  for 
authenticating  users.  In  theory,  traditional  passwords  are  very  secure;  given 


that  they  meet  certain  criteria  (e.g.  randomness,  minimum  length,  and  include 
5  capital  letters,  numbers,  and  special  characters).  The  additional  restrictions 
increase  the  password  strength  by  increasing  size  of  the  password  space — the 
total  number  of  possible  passwords,  which  significantly  increases  the  expected 
number  of  random  guesses  required  from  a  brute  force  attack.  In  truth,  the  more 
random  the  password,  the  better.  However,  a  complex  password  may  result  in 
10  unanticipated  consequences: 

In  reality,  users  end  up  breaching  the  security  measures  taken  by  these  pass¬ 
word  systems  in  one  of  two  ways:  1)  constructing  a  password  that  is  too  complex 
to  remember,  or  2)  ignoring  the  protocols  completely,  and  creating  a  simple  pass¬ 
word.  The  first  scenario  is  typical  of  a  person  who  creates  a  sufficiently  complex 
15  text-based  password,  but  writes  their  username  and  password  down  (or  alterna¬ 
tively  saves  their  login  information  in  a  text  file  called  “passwords”).  Thus,  the 
security  measures  taken  are  essentially  nullified.  The  second  case  occurs  when 
the  security  measures  are  ignored.  This  case  leaves  the  system  vulnerable  to  a 
brute  force  attack  because  the  password  space  is  reduced.  Therefore,  traditional 
20  password  mechanisms  are  not  necessarily  as  secure  as  they  appear. 

In  recent  years,  there  have  been  many  advancements  in  the  type  and  num¬ 
ber  of  user  authentication  methods,  including  gestures  and  patterns,  picture- 
based  passwords,  and  biometric  recognition  schemes.  Each  approach  attempts 
to  optimally  balance  security — preventing  unauthorized  users  from  accessing  the 
25  system — and  usability — granting  access  to  authorized  users  with  ease.  The  two 
extremes  for  such  systems  are:  1)  perfect  security,  which  does  not  grant  access 
to  anyone,  and  2)  perfect  usability,  which  grants  access  to  all  users.  Neither 
case  is  desired.  Ideally,  the  only  person  capable  of  gaining  access  should  be  the 
genuine  user. 

30  In  this  paper,  a  new  authentication  approach,  which  uses  a  sketch  (or  draw¬ 
ing)  with  biometric  information  as  the  form  of  authentication,  is  presented. 
The  difficulty  with  using  sketch-based  passwords  is  the  stochastic  nature  of  the 
drawing  process.  The  matching  algorithm  must  be  robust  enough  to  handle  the 
variations  from  the  genuine  user  and  simultaneously  be  secure  enough  to  reject 
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35  any  forgeries  (random  or  skilled). 

Traditional  text-based  passwords  typically  use  the  password  space  (based  on 
certain  assumptions)  as  the  measure  for  security.  However,  because  password 
comparisons  are  more  difficult  with  sketches  than  with  character  strings,  the 
password  space  in  this  case  is  not  a  good  measure  of  security.  Therefore  we 
40  provide  both  theoretical  and  empirical  evidence  that  such  a  password  system 
balances  security  and  usability. 

First,  a  review  (Section^  of  previous  research  related  to  graphical  passwords 
and  biometric  systems  is  discussed.  This  brief  survey  provides  a  summary  of 
alternative  graphical  passwords  and  biometric  systems.  In  Section[3j  the  sketch- 
45  based  password  system  is  discussed,  including  enrollment  and  login  phases.  In 
Section  [I]  we  introduce  and  discuss  fundamental  theory  about  the  sketch-based 
password  system,  including  complexity  analysis,  model  uniqueness,  and  fuzzi¬ 
ness  (or  tolerance).  The  experiments  and  results,  in  Section  [5j  demonstrate 
robustness,  usability,  and  security  of  the  sketch-based  authentication  system. 
50  Lastly,  conclusions  and  future  work  are  discussed  in  Section  [6] 

2.  Related  Work 

Many  researchers  have  studied  and  proposed  various  alternatives  to  text- 
based  passwords,  including  using  drawings,  patterns,  gestures  mim  pictures 
and  faces  00,  and  local  points  or  regions  Esnaig.  Additionally,  biometric 
55  recognition  methods,  such  as  fingerprint,  voice,  and  facial  recognition  systems 
have  been  recently  used  for  authenticating  user  access. 

2.1.  Graphical  Passwords 

The  concept  of  using  graphical  passwords  has  been  around  for  more  than  a 
decade.  Graphical  passwords  belong  to  one  of  three  groups  (as  proposed  in  0): 

so  1.  Drawmetric — recall-based  methods 

2.  Cognometric — recognition-based  methods  (also  referred  to  as  search  met¬ 
ric  [ID]) 
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3.  Locimetric — cued-recall  based  methods. 


Drawmetric  systems  require  users  to  create  a  distinctive,  yet  memorable, 
65  drawing  for  a  password  during  enrollment.  Then,  at  login  time  users  are  asked 
to  remember  their  password  and  accurately  reproduce  it.  This  type  of  system 
involves  a  complex  human-computer  interaction,  which  includes  the  conversion 
from  a  recalled  version  of  the  drawing  (stored  in  the  temporal  lobe  of  the  brain) 
to  the  fully  digital  representation  (stored  in  a  computer’s  or  device’s  memory). 
70  Therefore,  a  user  must  physically  draw  their  “password”  using  a  digital  tablet 
or  tablet  computer  (any  device  with  an  appropriate  user  interface  for  drawing). 

The  tablet  computer  market  explosion  has  greatly  increased  the  potential  for 
drawmetric  passwords  (or  password  managers).  Until  now,  drawmetric  systems 
have  not  been  able  to  reach  their  full  potential. 

75  Probably,  the  most  notable  drawmetric  system  is  Draw-A-Secret  (DAS)  (11). 
The  DAS  approach  uses  a  coarse  grid,  which  is  displayed  to  the  user,  to  encode 
the  drawing.  The  drawing  is  encoded  using  the  grid  cells.  The  ordered  sequence 
of  grid  cells  that  the  drawing  enters  produces  a  password  string.  Thus,  the 
matching  procedure  for  DAS  is  virtually  no  different  from  text-based  passwords 
so  (except  for  the  fact  that  the  string  is  generated  from  a  drawing);  the  string 
produced  at  login  time  is  compared  with  the  string  stored  during  enrollment. 

There  are  some  extensions  to  DAS  which  have  improved  performance.  For 
example,  Background  Draw-A-Secret  [2j  improves  performance  by  adding  back¬ 
ground  images.  Yet  Another  Graphical  Password  (YAGP)  [12]  improves  per- 
85  formance  by  using  a  finer  grid  than  the  original  DAS  approach.  The  result 
reported  include  100%  of  genuine  users  being  granted  access  and  only  1  security 
breach  in  study  using  18  participants.  By  removing  the  visible  grid  for  drawing 
and  including  stroke  color  as  an  additional  feature,  Passdoodle  mm  improves 
performance;  [14]  achieves  around  98%  accuracy  on  a  set  of  10  users. 

90  One  drawback  to  DAS  and  similar  approaches  is  the  difficulty  of  encoding 
near  intersection  points  of  grid  cells.  To  alleviate  this  problem,  the  Pass-Go 
[15]  approach  uses  grid  intersection  points  as  anchor  points  instead.  Thus, 
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the  encoding  records  an  ordered  sequence  of  horizontal,  vertical,  and  diagonal 
strokes. 

95  Cognometric  systems,  instead  of  drawing,  demand  that  users  recognize  and 
select  the  correct  set  of  images  from  a  randomly  assorted  set  of  images.  These 
types  of  systems  leverage  the  fact  that  humans  tend  to  recognize  something 
easier  than  they  can  remember  something.  Although  cognometric  system  still 
require  the  interaction  between  a  person  and  a  computer,  in  this  case,  the 
100  procedure  is  less  complicated  because  the  user  is  not  required  to  produce  a 
drawing.  Instead,  users  are  required  only  to  indicate  the  recognition  of  a  face, 
landscape,  or  any  other  image,  using  a  mouse,  keyboard,  or  touch  screen. 

The  best  example  of  a  recognition-based  system  is  PassFaces  [5].  During 
enrollment,  the  user  is  presented  with  a  random  set  of  faces  and  asked  to  select 
105  a  subset  of  them  to  be  their  “password.”  At  login  time,  the  user  is  presented 
with  the  correct  set  of  faces  dispersed  among  a  random  set  of  other  faces.  The 
user  is  then  asked  to  identify  the  correct  set  of  faces. 

Instead  of  faces,  the  Deja  Vu  |4j  system  uses  a  set  of  images  with  patterns 
and  art.  Studies  pTBJ  Hzj  have  shown  that,  even  with  choosing  faces,  people 
no  are  predictable  in  their  choice  of  password.  Deja  Vu  attempts  to  alleviate  this 
problem  by  making  the  images  very  randomized,  so  that  a  bias  or  preference 
toward  any  particular  image  is  less  likely  to  occur. 

Locimetric  systems,  instead  of  requiring  users  to  remember  entire  images, 
have  users  recall  and  identify  specific  points  or  regions  within  an  image.  The 
ns  idea  is  to  reduce  the  amount  of  information  that  a  user  is  required  to  remember. 
Given  a  larger  image,  a  user  selects  an  ordered  sequence  of  points  or  regions 
located  within  the  image.  In  principle,  the  image  should  help  stimulate  the 
process  of  remembering  the  points  previously  chosen,  hence  the  term  cued-recall. 

The  most  prominent  cued-recall  based  method  is  PassPoints  mmm.  Pass- 
120  Points  presents  the  user  with  a  single  image,  typically  having  numerous  land¬ 
marks:  people,  buildings,  objects,  etc.,  from  which  the  user  is  supposed  to  select 
5  different  locations  during  enrollment.  At  login  time,  the  user  is  again  presented 
with  the  same  image,  and  the  user  must  choose  the  correct  set  of  points  within 
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the  image  in  precisely  the  same  order. 

125  An  extension  to  the  PassPoints  philosophy,  called  Cued  Click-Points  |18j  [19] , 
presents  the  user  with  multiple  images  (one  at  a  time).  In  each  image  the  user 
selects  only  one  point  to  use.  Similarly,  the  ordered  sequence  of  locations  in  each 
image  becomes  the  password.  If  any  one  point  is  incorrect,  access  is  denied. 

Many  of  the  methods  using  drawmetric,  cognometric,  and  locimetric  pass- 
130  word  systems  are  compared  and  contrasted  in  [20].  This  literature  survey  ana¬ 
lyzes  the  multitude  of  methods  in  terms  of  both  usability  and  security,  especially 
the  vulnerabilities  to  different  attacks  (e.g.  brute  force,  phishing,  and  shoulder 
surfing) . 

2.2.  Biometric  Systems 

135  Biometric  systems — systems  that  measure  and  analyze  certain  biological 
properties  for  the  purposes  of  identity  recognition  m  — are  steadily  becoming 
a  part  of  everyday  life,  including  in  commercial  access  systems,  personal  com¬ 
puters,  and  smart  phones.  Biometrics  are  increasingly  popular  because  they 
are  supposed  to  be  universal,  distinct,  permanent,  and  collectible  [22 ]  prop- 
140  erties  that  uniquely  identify  an  individual.  For  these  reasons,  biometrics  are 
considered  much  more  secure  than  traditional  passwords. 

Biometrics  include  biological  properties  such  as  fingerprints,  voices,  faces, 
and  even  handwriting.  Fingerprints  have  been  widely  used  and  accepted  in 
forensic,  commercial,  government  security,  and  consumer  applications.  Despite 
145  the  fact  that  fingerprints  are  essentially  unique  to  every  individual,  there  are  still 
concerns  in  terms  of  a  system’s  ability  to  recognize  fingerprints  (especially  with 
consumer  applications).  For  example,  fingerprint  recognition  is  very  sensitive  to 
noise  and  partial  occlusion.  In  most  cases,  the  signal-to-noise  ratio  (SNR)  is  low, 
which  increases  the  difficulty  of  recognizing  fingerprints  accurately  (implying  the 
150  possibility  of  a  breach  in  security). 

Occlusion  also  presents  a  problem  for  fingerprint  recognition  because  the 
fingerprint  (and  its  features)  must  be  visible  (at  least  mostly  visible)  in  order  to 
determine  with  sufficient  confidence  that  it  matches  another  fingerprint.  There 
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are  several  reasons  for  occlusion  of  a  fingerprint  including  varying  poses  and 
155  pressure. 

Voice  recognition  systems,  although  not  as  widely  used  as  fingerprint  recog¬ 
nition  in  security  applications,  are  used  in  applications  such  as  voice-to-text  in 
cars  (e.g.  Ford  Sync)  and  smart  phones  (e.g.  Apple’s  Siri),  and  voice  command 
systems  (e.g.  Microsoft  Xbox).  There  are  many  reliability  concerns  when  it 
wo  comes  to  voice  recognition  systems  for  user  authentication.  Multiple  variables, 
such  as  distance  to  microphone(s) ,  background  noise,  and  vocal  anomalies  (e.g. 
colds,  dialects,  and  accents),  affect  the  reliability  of  a  voice  recognition  system. 
The  most  disturbing  of  these  variables  are  the  anomalies  because  they  naturally 
occur  in  human  voices,  but  voice  recognition  systems  still  experience  difficulty 
K5  in  handling  these  effects. 

Facial  recognition  systems  for  authenticated  access  are  also  becoming  more 
common  because  of  advancements  with  this  technology.  However,  there  are 
still  some  concerns  about  the  reliability  of  these  systems.  Facial  recognition 
most  commonly  experiences  difficulties  with  occlusion.  Faces  can  be  occluded 
no  because  of  clothing  or  accessories,  lighting  conditions,  or  even  pose.  As  with 
fingerprints,  the  confidence  of  the  face  recognition  is  reduced  when  there  is  oc¬ 
clusion.  Under  normal  operating  conditions,  one  would  expect  facial  recognition 
to  perform  quite  well.  However,  there  are  legitimate  security  concerns  for  false 
acceptances — when  the  system  grants  access  to  an  unauthorized  person. 

175  Although  it  is  difficult,  under  normal  conditions,  to  change  or  alter  a  person’s 
fingerprint,  voice,  or  face,  it  certainly  plausible  that  an  authorized  user  may  be 
coerced  (with  extreme  force)  into  unlocking  the  system,  thus,  granting  access 
to  unauthorized  users.  In  addition,  a  fingerprint  can  be  reproduced  (or  in  the 
extreme  case  a  finger  can  be  severed),  a  voice  may  be  recorded  or  copied,  and 
iso  a  face  recognition  system  can  be  fooled  with  a  picture  of  a  face.  Each  of  these 
measures  requires  going  to  great  lengths,  but  all  are  reasonable  means  to  fool  a 
biometric  system.  For  sketch-based  passwords,  we  believe  that  a  coerced  user  is 
less  likely  to  reproduce  a  sketch  accurately  enough  to  gain  access  to  the  system 
because  of  additional  nervous  jitter  while  under  duress.  However,  this  is  not 
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considered  in  this  work. 

According  to  Jain  et.  al,  “foolproof  personal  recognition  systems  simply  do 
not  exist  and  perhaps,  never  will”  [22) .  However,  in  this  paper,  we  attempt  to 
introduce  a  new  system  that  bridges  the  gap  between  security  and  usability. 

3.  Recognizing  Sketch-based  Passwords 

In  this  section,  the  biometric  sketch-based  password  system  is  discussed. 
The  method  used  for  recognizing  sketch-based  passwords  with  biometric  infor¬ 
mation  in  essence  works  by  first  constructing  a  descriptor  or  a  model,  and  then 
applying  a  matching  procedure  that  looks  for  consistency  between  a  sketch  and 
this  model. 

In  password  systems,  typically  the  terms  recognition  and/or  matching  are 
implied  to  mean  verification — the  problem  of  confirming  that  a  user  is  indeed 
who  he/she  claim  to  be — which  is  also  the  implication  in  this  paper. 


3.1.  Overview 


Figure  1:  The  diagram  depicts  the  primary  components  of  both  the  enrollment 
and  login  phases  for  the  proposed  sketch-based  password  system.  The  inputs 
(depicted  with  arrows)  to  the  “Scale  Normalization”  blocks  for  both  enrollment 
and  login  represent  the  input  sketches  from  the  user  for  the  respective  phases. 


There  are  two  primary  phases  which  are  used  in  a  sketch-based  password 
200  system: 

1.  Enrollment — User  registers  using  a  username  and  sets  his/her  sketch- 
based  password. 

2.  Login — User  enters  his/her  username  and  draws  an  instance  of  the  pass¬ 
word. 

205  Fig.  [T| outlines  the  major  procedures  necessary  for  both  enrollment  and  login 
phases.  The  diagram  shows  two  important  steps  involved  within  each  phase: 
capturing  the  input  sketches  and  normalizing  the  scale.  The  next  step  for  en¬ 
rollment  is  constructing  the  model  for  each  normalized  sketch,  from  which  a 
representative  model  for  the  class  of  acceptable  sketch-based  passwords  is  pro- 
210  duced.  This  representative  model  is  then  stored  in  the  user  database. 

After  normalizing  the  input  sketch  during  the  login  phase,  the  model  for  the 
claimed  user  identity  is  retrieved  from  the  database.  Then,  the  consistency  be¬ 
tween  the  model  and  the  sketch  are  determined  using  an  accumulative  method. 
If  enough  evidence  exists  for  the  sketch  to  be  considered  similar  to  the  exemplar 
215  sketches  that  constructed  the  model,  then  access  is  granted.  Otherwise,  access 
is  denied. 

3.2.  Input  and  Normalization 

The  input  of  the  sketch-based  password  system  is  simply  a  sketch.  More 
rigorously,  a  parameterized  sketch  (excluding  biometrics  for  now)  is  considered 
220  to  be  a  continuous  mapping  at:  I  R2,  expressed  as  a(s)  =  (x(s),  y(s ))  for  arc 
length  s  £  /,  where  I  is  the  interval  (0, 1)  without  loss  of  generality. 

Given  a  sketch,  the  scale,  according  to  Kendall’s  [23]  definition,  is  propor¬ 
tional  to  the  average  distance  from  the  center  of  gravity.  In  order  to  achieve  scale 
invariance,  every  sketch  is  normalized  to  unity  scale.  Therefore,  the  recognition 
225  does  not  depend  on  the  size  of  the  sketch.  Note  that  this  particular  definition 
of  scale  is  only  useful  for  non-occluded  shapes  or  curves.  In  the  application 
of  sketch-based  passwords,  occluded  sketches  are  considered  to  be  different  or 
incomplete  passwords.  Therefore,  occlusion  is  not  considered  in  this  paper. 
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3.3.  Enrollment 


230  During  enrollment,  the  user  is  expected  to  provide  multiple  instances  of  the 
sketch-based  password.  Multiple  sketches  are  required  in  order  to  construct  a 
model  which  is  more  accurate  and  robust  than  a  model  produced  by  a  single 
sketch.  The  main  idea  is  to  capture  more  of  the  possible  variations  from  a 
particular  user,  which  helps  to  produce  a  model  that  is  consistent  with  sketch- 
235  based  passwords  generated  by  that  particular  user.  This  is,  in  fact,  similar 
to  traditional  text-based  schemes.  In  most  text-based  password  systems,  users 
are  required  to  enter  their  desired  password  at  least  twice.  Therefore,  it  is  not 
unreasonable  for  a  sketch-based  password  system  to  demand  multiple  sketches 
during  enrollment.  Due  to  the  additional  complexity  of  a  sketch  compared  with 
240  an  alphanumeric  string,  it  is  reasonable  that  a  sketch  would  require  more  than 
two  examples  for  enrollment  (e.g.  3-5). 

Given  a  normalized  sketch,  the  goal  is  to  construct  a  model  or  descriptor 
that  captures  the  desired  properties  of  a  sketch,  which  include: 

•  shape  (e.g.  relative  distance  and  curvature) 

245  •  drawing  direction  (e.g.  sketch  tangent  direction) 

•  biometrics  (e.g.  pressure,  velocity,  or  acceleration) 

•  parametrization  (e.g.  time  or  arc  length). 

The  model  we  have  chosen  is  a  biometric  extension  and  generalization  of  a 
shape  recognition  approach  called  Simple  K-Space  (SKS)  [2HI22]-  A  general- 
250  ization  of  SKS  is  implemented  because  it  has  been  shown  to  be  robust  to  the 
multitude  of  variations  to  shape  contours  [22| .  Since  a  sketch  for  all  intents  and 
purposes  is  considered  to  be  a  shape,  an  extension  to  SKS  which  models  shape 
and  other  properties  seemed  appropriate. 

Before  discussing  the  model  construction,  several  local  features  of  the  sketch 
255  are  required.  These  features  are  intended  to  capture  the  local  properties  dis¬ 
cussed  above. 
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Two  shape  functions,  pa{s)  and  Ka(s),  are  computed  (or  estimated)  from 
the  normalized  sketch,  a(s).  pa(s)  =  ||a(s)  —  x0||  is  defined  as  the  Euclidean 
distance  between  the  spatial  coordinates  at  arc  length  s  along  the  sketch,  cc(s), 
260  and  an  arbitrary,  but  constant  spatial  reference  point,  Xq.  In  this  approach, 
we  have  chosen  the  reference  point  of  the  sketch  to  be  the  center  of  gravity. 
Ka(s)  is  defined  as  the  local  curvature  at  the  point  s  on  the  sketch.  Curvature 
is  defined  as  the  magnitude  of  the  derivative  to  the  unit  tangent  vector,  or 
k(s)  =  |T'(S)|  =  |a"(*)|. 

265  Typically,  pa(s)  €  U  and  Ka(s)  €  K,  where  U  C  R  is  defined  by  [0,  pmax } 
and  K  C  K.  is  defined  by  [0,  nmax].  In  practice,  pmax  and  Umax  are  determined 
respectively  by  the  length  of  the  diagonal  of  the  active  drawing  area  (x-y  plane) 
and  the  device  resolution. 

Note  that  both  features,  distance  and  curvature,  are  invariant  to  translation, 
270  rotation,  and  reflections. 

In  order  to  capture  the  drawing  direction  at  each  point  along  the  sketch, 
a(s),  the  tangent  direction  to  the  sketch  is  computed.  The  tangent  direction 
at  a  point  s  along  the  sketch  is  represented  by  a  scalar  angle  0a(s)  €  0,  where 
9a(s)  =  atan2(y,(s),a;,(s))  and  0  is  defined  by  the  interval  ( — 7r,  7r] . 

275  Since  sketches  drawn  by  most  users  are  very  noisy  (in  terms  of  jitter  caused 
by  shaking  of  the  hand),  9a(s)  is  significantly  impacted  by  the  amount  of  jitter. 
There  are  two  ways  to  handle  jitter  in  the  drawing  direction:  1)  quantizing  the 
direction  angle,  and  2)  smoothing  the  jitter  using  a  convolution  kernel.  In  our 
approach,  we  use  the  latter  approach  because  noise  near  a  quantization  level 
280  boundary  introduces  significant  errors.  The  direction  feature  is  not  smoothed 
directly,  however,  SKS  includes  a  parameter  that  blurs  all  features  (not  just 
direction)  in  order  to  be  more  robust. 

The  next  property  of  a  sketch  that  is  modeled  is  the  biometric  component — 
pressure — which  is  denoted  by  the  function  ba(s).  The  pressure,  ba(s)  €  B ,  is 
285  a  scalar  measurement  representing  the  amount  of  force  applied  by  the  pen  with 
the  active  area  on  the  device’s  screen,  where  B  is  defined  as  the  interval  [0,  1]. 

Finally,  the  arc  length,  sa(t)  =  fg  ||a(u)||  du,  is  used  as  a  feature.  The 
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arc  length  is  used  primarily  to  impose  a  general  ordering  of  the  samples  of 
the  sketch  (i.e.  how  a  sketch  is  drawn).  However,  it  also  significantly  improves 
290  performance.  The  arc  length  without  loss  of  generality  is  on  the  interval  S, 
defined  by  [0,  1]. 

After  computing  each  of  the  functions:  pa(s),  fta(s),  9a(s),  ba^s),  and  sa(t), 
the  model  of  a  sketch,  a(s ),  is  defined  as: 


ma(v)  =  *  J  exp  ^-i(v  -  va(r))TS  1(v-vce(r))^  dr 


(1) 


where  vq(t)  =  [pa{j)  Ka{r)  9a(r)  ba(r)  sa(r)]T  is  the  local  feature  vector  at 
the  point  r  on  the  sketch  (and  r  is  just  a  “dummy”  arc  length  variable),  z  is 


a  normalization  factor  (see  Section  3.5),  and  £  is  the  covariance  matrix  used 
295  for  smoothing  the  model.  The  intent  behind  blurring  the  model  is  to  make  the 
matching  of  a  sketch  more  robust  during  the  login  phase  (Section  |3.4[).  This  is 
discussed  further  in  Section  14731 

In  this  work,  £  is  assumed  to  be  a  diagonal  covariance  matrix  that  is  con¬ 
stant  over  all  sketches.  The  non-zero  parameters  in  the  matrix  are  determined 
300  experimentally.  However,  in  general,  a  diagonal  £  is  not  necessary. 

The  model  in  Eq.  Q,  ma :  J-  — >■  R,  is  a  scalar  function  defined  for  all 
vectors  v  =  [ p ,  n,  9,  b ,  s]T  £  J7,  where  BcUxKxQxBxS.  The  scalar 
value  represents  a  likelihood  that  there  exists  a  point,  at  an  arc  length  of  s  on 
a ,  that  is  a  distance  of  p  from  the  reference,  has  a  curvature  of  k,  has  a  drawing 
305  direction  of  9 ,  and  has  a  pressure  of  b  . 

The  model,  in  fact,  may  be  viewed  as  a  hyper-surface  defined  over  JcK5, 
and  it  very  much  resembles  a  kernel  density  estimator  m-  However,  the  main 
difference  is  the  normalization  factor,  z.  Density  estimators  normalize  such  that 


/ma(v)  dv  =  1, 

instead,  an  alternative  method  of  normalization,  which  is  better  suited  to  this 
application,  is  used  (Section |3.5|. 

After  constructing  each  model  for  the  multiple  sketches  drawn  by  the  user 
during  the  enrollment  stage,  it  is  useful  to  construct  a  single  representative 


12 


310 


model  (opposed  to  matching  with  all  exemplars).  This  is  accomplished  by  av¬ 
eraging  the  set  of  models.  At  first  this  approach  may  appear  to  be  trivial, 
however,  the  reason  behind  it  is  not  intuitive.  The  assumption  is  that  sketches 
input  by  the  user  during  enrollment  must  be  similar  sketches.  This  means  the 
sketches  must  be  of  the  same  shape  and  drawn  in  a  similar  manner.  Therefore, 
315  on  the  manifold  of  sketches,  they  are  considered  to  be  “close”  in  term  of  their 
geodesic  distance.  Since  the  models  are  constructed  from  these  sketches,  they 
too  lie  in  “close”  proximity  on  the  manifold  of  models.  The  average  is  then  an 
excellent  representative  model  for  the  class  because  manifolds  are  locally  linear 
and  the  average  optimally  minimizes  the  distance  from  each  point  in  a  linear 
320  space.  However,  if  a  user  enters  extremely  different  sketches  during  enrollment 
then  the  average  model  is  a  poor  representative  of  the  set  of  sketches.  A  simple 
outlier  test  to  determine  if  the  sketches  provided  by  the  user  during  enrollment 
are  sufficiently  different  is  implemented. 

An  interesting  observation  is  that  the  optimal  sketch  necessary  to  produce 
325  the  representative  model  is  unknown  to  the  user  after  enrollment,  unless  the 
user  is  perfectly  consistent  (practically  impossible).  However,  as  individuals  use 
this  system  by  logging  in  with  a  sketch,  they  adapt  and  learn  what  the  system 
expects  to  be  a  “correct”  sketch,  which  is  significantly  easier  if  one  is  the  genuine 
user  and  already  know  what  was  drawn  and  how  it  was  drawn.  Although  this 
330  not  the  primary  focus  of  this  paper,  it  is  an  interesting  process  that  occurs 
during  the  human-computer  interaction. 

Now  that  the  enrollment  phase  has  been  discussed,  we  move  on  to  discuss 
the  login  phase  which  is  the  primary  mode  of  operation. 

3-4-  Login 

335  During  the  login  phase,  a  user  inputs  his/her  putative  identity  using  a  user- 
name  or  pin,  and  then  the  user  draws  the  sketch-based  password  (attempting 
to  replicate  the  underlying  biometric  signature  used  during  enrollment).  This 
sketch  is  then  determined  to  be  either  sufficiently  or  insufficiently  consistent 
with  the  representative  model  of  the  putative  user. 
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The  method  used  for  computing  this  measure  of  consistency  is  a  higher  di¬ 
mensional  extension  of  the  matching  procedure  used  by  SKS.  The  SKS  approach 
uses  an  accumulative  framework  for  determining  the  consistency  of  a  shape  with 
a  shape  model.  Thus,  the  approach  used  here  is  very  similar  for  sketches. 

Given  a  login  sketch,  ate(s)  =  (xe(s),  ye(s))  (with  corresponding  curvature, 
direction,  pressure,  and  arc  length  functions)  and  model  ma(v),  the  accumula¬ 
tor  is  defined  as 


where 


vat(r,x)  =  [||x-c^(t)||  Kae(r)  9ae(r)  bae(r)  sae(r)]T 


is  the  feature  vector  defined  by  a c  and  x;  A(x)  represents  the  likelihood  that  x  = 
345  (x,  y)  is  the  reference  point  of  the  sketch,  a. a.  This  likelihood  may  be  considered 

to  be  the  result  from  a  path  integral  over  the  hyper-surface  of  the  model,  ma, 
where  the  path  is  defined  by  x  and  the  local  features  of  the  login  sketch.  An 
example  of  this  path  integration  using  a  2D  model  (higher  dimensional  cases 
are  more  difficult  to  illustrate)  is  shown  in  Fig.  [2j 


Figure  2:  The  surface  represents  the  model  constructed  over  the  p-s  feature 
space,  and  the  line  represents  an  integration  path  defined  by  x  and  the  features 
of  otg.  The  result  from  each  integral  path  represents  the  likelihood  that  x  is  the 
reference  point  for  the  model  sketch. 
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350  Assuming  that  a  and  a.(  are,  indeed,  two  similar  sketches,  the  consistency 
measure  increases  as  x  approaches  the  actual  reference  point  of  a^(s).  The 
result  is  a  global  maximum  in  the  accumulator  at  the  location  of  the  “best” 
reference  point.  On  the  other  hand,  if  a  and  a n  are  sufficiently  different,  no 
significant  peak  will  occur.  However,  the  accumulator  may  have  other  local 
355  maxima;  their  amplitudes  are  usually  small  in  comparison  to  that  of  a  peak 
generated  from  a  similar  sketch.  Fig.  [3]  shows  the  difference  between  matching 
and  non-matching  accumulators. 

i 

0.8 

0.6 

0.4 

0.2 

0 

X 

(b) 

Figure  3:  Example  of  an  accumulator  that  indicates  a  sketch  (shown  on  top  of 
the  accumulator)  that  is  consistent  with  the  model  (a) ,  and  an  accumulator  that 
indicates  a  different  sketch  (also  shown)  is  inconsistent  with  the  same  model  (b). 

Since  the  amplitude  at  every  point  in  the  accumulator  represents  the  likeli¬ 
hood  measure  for  that  point  being  the  reference  point,  then  the  best  reference 
point  occurs  at  x*  =  argmaxA(x).  Thus,  the  authentication  decision  is  made 

X 

as  follows: 

{Access  Granted,  A(x*)  >  Thresh 
Access  Denied,  A(x*)  <  Thresh 

which  means  that  if  the  login  sketch,  ot£,  is  sufficiently  consistent  (the  accumu¬ 
lator  peak  is  greater  than  than  the  threshold,  Thresh )  with  the  model  for  a, 
360  then  the  two  sketches  are  considered  to  be  similar. 
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3.5.  Model  Normalization 


Normalizing  the  model,  although  not  a  necessity,  makes  determination  of 
sketch  consistency  with  a  model  much  easier.  Consider  the  unnormalized  model 
(i.e.  where  z  =  1)  for  the  sketch  a(s) 


(3) 


The  sketch  which  is  most  consistent  with  rha  is  a(s).  Therefore,  the  normal¬ 
ization  factor 


(4) 


which  represents  the  result  of  integrating  on  the  path  (defined  by  the  best 
reference  point  x*)  over  the  unnormalized  model.  Using  this  definition  of  z, 
the  model  is  normalized  such  that  acca  a(x.*)  =  1  (i.e.  the  consistency  between 


365  the  model  of  a(s)  and  cc(s)  itself  is  maximal).  In  practice,  this  means  if  a 


sketch  is  consistent  with  the  model,  then  the  magnitude  of  the  peak  in  the 
accumulator  should  be  close  to  one.  Otherwise,  it  should  be  bounded  away 
from  one.  How  far  below  one  depends  on  the  severity  of  the  deformation  from 
the  original  sketch.  The  more  deformed  the  sketch,  the  smaller  the  peak  in  the 
370  accumulator.  Therefore,  normalizing  the  model  in  this  manner  simplifies  the 
process  of  determining  the  consistency  of  the  match. 

4.  Discussion  and  Analysis 

In  this  section,  various  properties  and  theorems  are  discussed,  including 
duality  and  computational  complexity  of  constructing  a  model,  uniqueness  of 
375  a  model,  and  fuzziness  in  the  accumulator.  Every  claim  introduced  here  is 
crucial  for  showing  that  the  sketch-based  password  system  is  robust,  secure, 
and  efficient. 

4-1-  Duality  and  Computational  Complexity 

One  important  property  of  SKS  is  the  duality  for  building  a  model  of  a 
380  sketch.  The  direct  implementation  as  expressed  in  Eq.  ([l])  is  called  the  primal 
form,  and  it  is  implemented  in  Algorithm  [l] 
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Algorithm  1  Primal 
for  v  £  T  do 

for  r  £  [0,  1]  do 

mQ(v)  •«—  mQ(v)  +  exp 
end  for 
end  for 


(-§  (v“  va(r))TS  *(v-v„(t))) 


There  also  exists  a  dual  to  the  primal  algorithm,  in  which  the  curve  is  only 
iterated  over  once  rather  than  multiple  times  (as  with  the  primal  form).  The 
dual  is  outlined  in  Algorithm  [2] 

Algorithm  2  Dual 
for  r  G  [0,  1]  do 
for  v  £  J  do 

ma(v)  «-  mQ(v)  +  exp  (-§  (v  -  v„(r))T  XT1  (v  -  va(r))) 

end  for 
end  for 


385  Both  primal  and  dual  algorithms  implement  the  same  integrals,  but  the  man¬ 
ner  in  which  the  integrals  are  computed  distinguishes  them  from  one  another. 
The  primal  form  computes  the  model  by  integrating  over  the  sketch  for  every 
point,  v,  in  the  model.  However,  the  dual  only  iterates  over  the  model  for  every 
point  along  the  sketch.  Note  the  subtle  distinction  and  that  both  primal  and 
390  dual  have  exactly  the  same  algorithmic  complexity.  However,  the  dual  can  be 
used  to  construct  an  approximate  model  more  efficiently,  as  we  explain  below. 

The  dual  form  (Algorithm  [2])  provides  a  way  to  efficiently  construct  a  good 
approximation  of  the  model.  For  each  point  on  the  sketch,  every  point  v  is 
updated  in  the  model  by  a  factor:  exp  \  (v  —  va(r))T  £_1  (v  —  va(r))^ . 
395  This  factor  represents  a  Gaussian  centered  at  some  location  in  the  model,  which 
is  defined  by  the  local  features  on  the  sketch,  denoted  by  va  (r) .  Therefore,  only 
points  in  the  model  that  are  within  a  reasonable  local  neighborhood  of  the  center 
of  this  Gaussian  need  updating,  which  significantly  reduces  the  computational 
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400 


complexity  for  computing  the  model.  Using  a  finite  region  of  support  for  the 
Gaussian,  we  can  show  that  dual  algorithm  scales  linearly  with  the  number  of 


points  along  the  sketch  ( Observation  [l]). 


Observation  1.  The  complexity  for  constructing  the  model  is  0(p),  where  p  is 
the  number  of  samples  along  the  sketch. 

Proof.  For  each  of  the  p  points  along  the  sketch,  every  point  inside  the  local 
405  neighborhood  of  the  corresponding  features  of  p  is  updated.  Thus,  the  compu¬ 
tational  complexity  is  0(pJ\fp)  where  Afp  denotes  the  size  of  the  neighborhood 
or  region  of  support.  Assuming  that  the  neighborhood  size,  defined  by  the  set  of 
points  inside  the  2S  or  3E  hyper-ellipsoid,  is  fixed  for  all  p ,  then  J\fp  is  constant 
and  the  complexity  scales  linearly  with  the  number  of  samples  over  the  sketch. 
4io  Therefore,  the  complexity  reduces  to  0{p).  0 

1^.2.  Model  Uniqueness  in  the  Limit 

An  interesting  result  of  using  the  SKS  model  (Eq.  0)  is  the  property  of 
having  a  unique  model  (in  the  limit).  This  comes  primarily  from  the  intuition 
behind  the  shape  recognition  model,  but  it  is  also  applicable  to  the  generaliza- 
415  tion  discussed  in  this  paper. 

Consider  the  limit  of  the  SKS  model  of  a  parameterized  curve  as  the  elements 
of  the  positive  definite  matrix  S  approach  0  (i.e.  X  — >  0  where  0  is  the  matrix 
with  all  zero  elements^ 


where 


s->°  V  ^  /  [0,  x^O 

Now,  consider  a  mapping  if:  S  — >  A4  (defined  by  the  model),  which  maps 
an  element  of  S  -the  “space  of  sketches” — to  the  “space  of  models,”  A4.  Note 

1  lim  X  =  lim  (S 


£->o  f->0 
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the  distinction  between  m„(v),  which  maps  an  element  of  JF  to  R,  and  ip,  which 
maps  a  “sketch”  to  some  hyper-surface  defined  by  (v,m„(v))  in  M. 

420  Here,  we  show  that  the  mapping  ip  is  a  one-to-one  mapping  in  the  limit, 
which  implies  the  smoothing  matrix  S  may  be  used  to  set  the  trade-off  between 
a  unique  model  or  tolerant  model  (i.e.  security  vs.  usability). 

Theorem  1.  In  the  limit  as  X  — ►  0,  two  models  ip(a.f)  and  il’(a2)  of  two 
sketches,  aii  and  a2  respectively,  are  equal  iff  a2  =  TiSO(a i),  where  TiSO  is 
425  some  isometry  (e.g.  translation,  rotation,  or  reflection). 

In  order  to  prove  Theorem  [T]  the  following  lemma  is  necessary. 

Lemma  1.  Given  that  the  feature  vector  v(s)  at  s  is  invariant  to  any  isometry, 
then  vi(s)=v2(s)  iff  a2  =  Tiso(a i). 

Proof.  First,  we  need  to  make  sure  that  the  assumption  about  the  feature  vectors 
430  holds.  In  the  case  for  sketches,  p  which  is  a  distance  from  the  center  of  gravity 
is  invariant  to  translation,  rotation,  and  reflection;  as  is  the  curvature  n.  The 
tangent  direction  9  is  not  directly  invariant  to  rotation  or  reflection.  However, 
the  angle  can  be  measured  relative  to  the  principle  direction  (which  rotates 
and  reflects  with  the  sketch).  Therefore,  9  may  also  be  invariant  to  isometries. 

435  The  biometric  b  is  also  invariant  to  isometries  because  it  translates,  rotates, 
and  reflects  with  the  sketch,  and  arc  length  is  also  invariant.  Therefore,  with 
sketches  we  may  ensure  that  the  features  are  invariant  to  any  isometry. 

Assuming  that  a2  =  TiSO(a. i),  then  due  to  the  invariance  to  isometries 
vi(s)=v2(s).  And  if  vi(s)=v2(s),  then  e*2  is  equal  to  (up  to  a  rigid  trans- 
440  formation) .  □ 

Now,  we  can  prove  Theorem  [l] 

Proof.  Assume  that  cc2  =  TiSO(a i).  From  Lemma] l]  this  implies  that  vi(s)=v2(s). 
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Therefore, 


1 


lim  ma,  (v) 

S-s-0 


1 

z 

1 

z 


-vW)  dr 

-v«2(t))  dr 


=  lim  mao  (v). 
s->o 


Thus,  ip{a-i_)  =  V’(Q!2)- 

Conversely,  we  want  to  show  that  ip  (qi)  =  ip  (a. 2)  implies  0:2 
However,  if  the  contrapositive: 


TiSo(ai). 


a-2  7^  Tiso(ai)  =>  ip  (ai)  7^  ip  (a2) 


is  true,  then  the  original  statement  itself  is  also  true. 

So,  assuming  that  a2  7^  TiSO(a  1),  then  we  know  that  vi(s)  7^  v2(s)  from 
LemmaJI]  Therefore,  3s*  s.t.  v*  =  Vi(s*)  7^  v2(s*).  Then,  it  follows  that: 

lim  ma  (v*)  =  1  [  d(v*  -  vQl(r))  dr 

s^o  z  J0 


1 0 

=  -(5(v*  -  Vcei(s*))  =  1 

2  Z 


and 

1  f1 

|mmQ2(v*)  =  -|  d(v*-vQ2(r))  dr 
=  yd  (v*  -  va2(s*))  =  0 

Thus,  ^(ai)  7-  ip(ot2). 


□ 


445  This  concept  of  uniqueness  does  not  theoretically  hold  for  arbitrary  S. 
However,  the  system  is  designed  intentionally  so  that  there  is  some  level  of 
tolerance — non-uniqueness — in  the  model.  This  is  due  to  the  fact  that  people 
are  not  perfect;  they  make  errors  when  reproducing  a  sketch.  So,  in  order  to 
make  the  system  more  robust  and  usable,  the  model  is  intentionally  fuzzy  and 
450  not  unique.  In  fact,  true  uniqueness  is  not  desirable. 

Intuitively,  we  are  claiming  that,  in  general,  uniqueness  is  only  a  local  prop¬ 
erty  of  the  model.  We  provide  some  experimental  evidence  to  support  this  claim 
in  Section  EU 
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4-3.  Robustness  in  the  Accumulator 

455  Given  a  model  ma(v),  there  exists  some  continuous  path  p  (Fig.  [2])  over  the 
model,  that  is  the  p  is  defined  in  terms  of  the  features,  that  maximizes  the  peak 
in  the  accumulator  (Eq.  §)•  In  principle,  this  path  is  defined  by  a.  Therefore, 
there  exists  a  small  “band”  or  “tube”  around  the  path  p  that  defines  allowable 
deviation  from  the  optimal  path  (corresponding  to  deviations  of  the  sketch)  over 
460  the  model,  which  results  in  a  “small”  difference  in  the  accumulator  peak. 

Primarily  due  to  jitter  and  other  noise  factors  while  drawing,  most  people 
cannot  perfectly  reproduce  their  own  sketch.  A  significant  (but  less  than  op¬ 
timal)  amount  of  accumulation  occurs  when  a  sketch,  corresponding  to  a  near 
optimal  path  over  the  model,  is  drawn.  Therefore,  there  is  some  amount  of 
465  tolerance  (i.e.  robustness)  built  directly  into  the  matching  procedure.  This  tol¬ 
erance,  which  is  also  controlled  by  £,  is  what  makes  this  system  more  practical. 
Thus,  more  blurring  in  the  model  implies  less  security  of  the  system  because 
less  accuracy  is  required  of  the  user. 

5.  Experiments  and  Results 

470  In  this  section,  we  demonstrate  the  feasibility  and  security  of  our  system 
through  experimentation.  In  order  to  demonstrate  the  current  level  of  perfor¬ 
mance,  several  experiments  are  performed  using  both  synthetic  and  hand-drawn 
sketches.  The  purpose  of  these  experiments  is  two-fold:  1)  to  support  the  claims 
in  Section [4]  and  2)  to  provide  a  performance  analysis  of  the  proposed  password 
475  system. 

In  this  paper,  as  well  as  many  other  biometric  systems,  performance  is  mea¬ 
sured  by  the  false  acceptance  rate  (FAR)  versus  the  false  rejection  rate  (FRR) 
curve.  The  FAR  is  the  number  of  accepted  forgeries  divided  by  the  total  number 
of  forgeries,  and  the  FRR  is  the  number  of  genuine  sketches  that  are  rejected 
480  divided  by  the  total  number  of  genuine  sketches.  In  some  cases,  performance  is 
reported  with  a  single  number  referred  to  as  the  equal  error  rate  (EER),  which 
is  the  rate  where  the  EAR  and  FRR  are  equal. 
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5.1.  Model  “ Uniqueness ” 

In  this  set  of  experiments,  we  demonstrate  that  the  SKS  model  is,  for  all 
485  intents  and  purposes,  unique.  The  idea  is  to  show  that  a  “small”  change  in 
the  sketch  results  in  a  “small”  change  in  the  model.  These  experiments  provide 
some  experimental  evidence  to  support  Theorem  [l] 

Given  two  distinct — drawn  by  different  individuals — sketches,  we  compute 
50  intermediate  sketches  using  linear  interpolation.  This  allows  us  to  consider  50 
unique  sketches  and  their  corresponding  models  from  one  sketch  to  another.  The 
measures  used  to  to  quantify  the  differences  between  the  models  and  sketches 
respetively  are  the  mean  squared  errors  (MSEs): 

MSEs(ai,oi2)  =  Y  (au  -  a2l)2  (5) 

i 

MSEM(rni1m2)  =  E  (my  -  m2j)2  .  (6) 

3 

We  can  observe  how  a  change  from  the  original  sketch  is  reflected  in  the 
models  by  plotting  the  MSE  of  the  models  (Eq.  ©)  with  respect  to  the  MSE  for 
490  the  sketches  (Eq.  ([5])).  In  this  case,  the  MSE  is  calculated  using  the  interpolated 
sketch  (model)  and  the  original  sketch  (model).  The  expectation  is  that  as  the 
interpolated  sketch  differs  from  the  original  sketch  (in  terms  of  MSE),  that  the 
MSE  between  the  models  will  also  increase. 

Some  examples  of  this  procedure  are  shown  using  projections  of  the  higher 
495  dimensional  models  in  Fig.  [4j  Notice  how  every  perturbation  of  the  sketch 
results  in  a  corresponding  change  in  the  model,  which  is  exactly  what  we  imply 
when  we  say  that  model  is  (locally)  unique.  Some  example  plots  of  model  MSE 
vs.  sketch  MSE  are  shown  in  Fig.  [5] 

5.2.  Robustness 

500  In  the  following  experiment,  we  demonstrate  the  robustness  of  the  accumu¬ 
lative  framework.  As  in  the  previous  section,  we  use  the  interpolation  between 
two  distinct  sketches.  While  holding  the  model  constant  (i.e. ,  let  the  model 
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Figure  4:  Shows  the  interpolation  between  a  heart  and  a  figure-8  (row  1).  Then, 
the  corresponding  projections  of  the  models  onto  the  p-s  plane  are  shown  in  row 
2.  Here,  we  can  see  that  the  first  two  shapes  (and  models)  are  more  similar  than 
the  first  and  the  last. 

be  constructed  from  the  first  in  the  interpolated  sequence),  let  the  login  sketch 
vary.  As  the  sketch  differs,  the  accumulator  peak  changes  too.  Initially,  the  dif- 
505  ference  between  the  accumulator  peaks  is  very  little.  As  the  sketch  differs  more 
and  more,  the  accumulator  peak  becomes  less  distinctive,  indicating  less  simi¬ 
larity  between  sketches.  Therefore,  despite  small  perturbations  (from  the  sketch 
producing  the  model)  occurring,  the  accumulator  will  still  exhibit  a  sufficiently 
larger  peak,  indicating  a  matching  sketch.  This  implies  that  the  matching  pro- 
510  cedure  is  intentionally  fuzzy,  which  makes  the  system  more  robust. 

A  sequence  of  test  shapes  and  the  accumulators  are  shown  in  Fig.  [6j 

5.3.  D00DB  Database 

In  this  set  of  experiments,  we  use  the  D00DB  database  [28)  29j,  which  con¬ 
tains  a  set  of  finger-drawn  doodles  (or  sketches)  and  pseudo-signatures.  For  the 
515  purposes  of  comparing  our  system  with  a  state-of-the-art  method,  we  outline 
our  experiments  in  the  same  manner. 
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MSE  in  model  space  vs.  MSE  in  sketch  space 


Figure  5:  The  plot  shows  how  the  MSE  in  the  “space  of  sketches”  is  reflected 
in  the  “space  of  models”  for  the  interpolation  from  one  sketch  to  4  different 
sketches.  Initially,  the  MSE  is  null  for  both  because  the  sketches  and  models  are 
identical.  However,  as  the  sketches  becomes  different,  the  models  also  become 
increasingly  different  which  is  reflected  in  the  examples  shown  here.  The  exact 
nature  of  the  curve  depends  on  both  starting  point  and  ending  point  in  the 
“space  of  sketches.” 

The  DooDB  database  contains  files  which  include  both  spatial  and  temporal 
information,  which  are  used  for  positional,  velocity,  and  acceleration  type  fea¬ 
tures.  Since  their  data  was  collected  using  a  device  with  a  resistive  touch  screen 
520  and  without  any  hardware  for  detecting  pressure,  they  do  not  use  pressure  as 
a  biometric  feature.  Therefore,  to  compare  the  systems  on  a  level  playing  field, 
our  system  is  tested  on  the  DooDB  database  without  the  biometric  pressure. 
We  will  demonstrate  the  advantage  of  using  biometric  pressure  in  Section  |5.4| 
The  performance  of  our  SKS-based  method  is  compared  with  the  method 
525  used  in  [35],  which  is  a  Dynamic  Time  Warping  (DTW)  based  system.  They 
test  performance  on  both  doodles  (or  sketches)  and  pseudo-signatures,  and  they 
report  an  EER  for  both  random  and  skilled  forgeries  (with  the  expectation  that 
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Figure  6:  The  test  sketch  is  overlaid  on  the  accumulator  in  order  to  show  how 
the  accumulator  changes  as  the  shape  is  deformed.  Notice,  how  even  deformed 
versions  of  the  same  sketch  reveal  a  peak  in  the  accumulator  (columns  1-4). 
However,  if  the  sketch  is  deformed  too  much  the  peak  is  diminished  (column  5). 

skilled  forgeries  will  have  worse  performance).  The  performance  measures  used 
are  the  random  forgery  EER  (denoted  as  EERrc[)  and  skilled  forgerj0  EER 
530  (denoted  as  EERsk)  for  both  doodles  and  pseudo-signatures. 

The  results  from  [29]  and  our  results  are  both  reported  in  Table  |T]  for  com¬ 
parison.  In  2S],  three  different  features  sets  are  used  with  the  DTW  method. 
These  features  are  ATVS-Pos,  ATVS-Vel,  and  ATVS-Acc,  which  represent  the 
spatial  coordinates,  velocity,  and  acceleration  features  respectively. 


2  A  skilled  forgery  is  defined  as  one  in  which  the  forger  knows  what  the  sketch  looks  like  as 
well  as  its  beginning  and  ending  points 
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Doodles 

Pseudo-signatures 

Method 

Session 

EER-rd 

EERsk 

EERrd 

EERsk 

ATVS-Pos 

1 

2.7 

28.0 

3.5 

28.6 

ATVS-Vel 

1 

3.4 

26.7 

1.6 

23.9 

ATVS-Acc 

1 

4.5 

28.1 

2.2 

19.8 

SKS 

1 

1.4 

28.0 

1.6 

23.3 

ATVS-Pos 

2 

7.6 

36.4 

5.0 

34.5 

ATVS-Vel 

2 

6.3 

33.9 

3.8 

29.7 

ATVS-Acc 

2 

7.3 

34.1 

4.3 

25.0 

SKS 

2 

3.8 

35.9 

5.2 

28.5 

Table  1:  Performance  comparison  between  DTW  and  SKS  on  DooDB  database, 
which  included  both  doodles  and  pseudo-signatures.  The  best  performance  is 
achieved  using  the  SKS-Bio,  which  is  the  approach  presented  in  this  paper. 

535  The  results  in  Table  [l]  show  that  even  without  biometric  pressure  SKS  is 
comparable  to  (better  than  in  some  cases)  the  DTW  method.  However,  there 
are  some  important  observations  to  note.  First,  SKS  appears  to  perform  slightly 
better  than  DTW  on  the  doodles,  but  not  on  the  pseudo-signatures.  Recognizing 
doodles  (or  sketches)  was  the  objective  of  our  approach,  and  we  have  achieved 
540  an  improved  results  on  the  set  of  doodles.  For  the  pseudo-signature,  we  still 
achieve  comparable  performance  to  the  DTW  approach,  despite  not  designing 
the  system  with  signatures  in  mind.  Second,  but  probably  most  significant,  is 
that  our  system  is  easily  generalized  to  incorporate  features  such  as  biometric 
pressure  as  a  property  of  a  sketch.  Since  the  DooDB  database  does  not  have 
545  pressure,  we  cannot  accurately  compare  the  two  systems  using  pressure.  How¬ 
ever,  we  believe  that  biometric  pressure  significantly  improves  performance  over 
those  systems  without  it.  Finally,  the  DTW  warping  approach  tests  both  ve¬ 
locities  and  accelerations.  In  our  experiments,  we  found  that  the  improvement 
from  velocity  came  more  from  the  direction  than  the  magnitude  (or  speed). 
550  Now,  acceleration,  which  is  a  second  derivative  with  respect  to  time,  is  a  very 
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noisy  feature.  In  some  cases,  acceleration  improves  performance  and  in  others 
acceleration  worsens  it. 

5-4-  Biometric  Pressure 

The  following  set  of  experiments  aims  to  demonstrate  the  potential  security 
555  benefit  from  adding  biometric  pressure  to  sketch-based  password  systems.  A 
total  of  100  sketches  (with  pressure  information)  from  5  different  users  were 
obtained  using  a  Samsung  Galaxy  Note  10.1.  Each  of  the  5  users  provided  10 
genuine  sketches  (i.e.  50  genuine  sketches  in  total),  and  the  remainder  of  the 
100  sketches  were  skilled  forgeries.  Similar  to  the  experiments  performed  on 
560  the  DooDB  database,  we  build  a  model  using  3-5  example  sketches  (the  first 
3-5  sketches  provided)  and  measure  the  verification  performances  (in  terms 
of  EERrd  and  EERsk )  on  the  remaining  sketches.  The  results  are  shown  in 
Tabled 


Method 

4j=-  Examples 

EERrd 

EERsk 

SKS 

3 

2.5 

17.8 

SKS-Press 

3 

3.5 

10.7 

SKS 

4 

2.5 

16.6 

SKS-Press 

4 

4.1 

3.7 

SKS 

5 

1.6 

14.0 

SKS-Press 

5 

0.0 

3.3 

Table  2:  Performance  comparison  between  the  sketch-based  password  system 
with  and  without  biometric  pressure.  Observe  how  the  pressure  makes  an  im¬ 
provement  in  performance  in  almost  all  cases:  all  skilled  forgery  scenarios,  and 
the  5  example  case  for  random  forgeries. 

The  performance  difference  between  SKS  without  pressure  (SKS)  and  SKS 
565  with  pressure  (SKS-Press)  is  substantial;  a  more  than  12%  reduction  in  EER. 
The  performance  improvement  for  the  random  forgery  scenario  is  less  significant 
than  that  of  the  skilled  forgery  scenario.  This  is  reasonable  because  in  the 
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random  forgery  case,  the  shape  and  directional  components  of  the  sketch  are 
already  mostly  distinct.  Thus,  there  is  not  much  to  improve  upon.  However,  for 
570  skilled  forgeries  the  shape  and  direction  components  are  as  similar  as  humanly 
possible.  Therefore,  we  can  attribute  the  performances  differences  between  SKS 
and  SKS-Press  to  the  biometric  pressure.  In  every  skilled  forgery  scenario  tested: 
3,  4,  and  5  example  sketches,  the  best  performance  is  achieved  by  the  addition 
of  pressure. 

575  6.  Conclusions 

In  this  paper,  we  discussed  a  novel  sketch-based  password  system  incorpo¬ 
rating  shape  and  the  biometric  pressure.  Using  an  extension  of  the  SKS  shape 
recognition  algorithm,  we  analyzed  both  the  security  and  usability  aspects  of 
the  approach.  It  was  demonstrated  that  the  model  of  a  sketch  is  unique  in  the 
58o  limit;  implying  a  perfect  security  scenario.  However,  it  was  also  shown  by  relax¬ 
ing  the  smoothing  matrix,  we  were  able  to  balance  both  security  with  usability. 
The  fuzziness  that  is  built  into  the  model  of  a  sketch  by  means  of  the  smoothing 
matrix  directly  affects  the  robustness  and  usability  of  authentication  system. 

The  system,  without  using  biometrics,  was  compared  with  a  state-of-the-art 
585  DTW  approach  using  the  same  database  of  pseudo-signatures  and  doodles  (or 
sketches).  In  general,  the  SKS  achieves  similar  performances  (better  in  some 
cases).  The  advantage  of  SKS  is  the  fact  that  it  is  more  general  than  DTW. 
DTW  requires  restrictions  such  as  boundary  and  monotonic  constraints,  where 
as  SKS  is  robust  to  the  start /end  points  and  does  not  impose  any  monotonicity 
590  constraints.  Thus,  freeing  SKS  to  accumulate  evidence  of  consistency  between 
sketch  and  model  by  means  of  more  general  transformations,  opposed  to  the 
constrained  method  used  by  DTW. 

More  importantly,  the  addition  of  biometric  pressure  was  shown  to  increase 
the  level  of  performance  by  more  than  12%.  In  security  sensitive  applications, 
595  the  results  demonstrate  the  potential  for  biometric  pressure  to  provide  improved 
security  compared  with  sketch-based  passwords  that  do  not  include  this  feature. 
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The  average  person  is  capable  of  reproducing  a  simple  sketch  with  biometric 
pressure  within  a  certain  degree  accuracy.  However,  there  are  still  many  ques¬ 
tions  to  answer  about  using  pressure  for  sketch-based  passwords. 

coo  6.1.  Future  Work 

In  the  future,  we  hope  to  construct  a  larger  dataset  that  consists  of  sketches 
with  biometric  pressure  in  order  to  further  demonstrate  the  security  and  us¬ 
ability  improvement  by  using  this  feature.  To  our  knowledge,  there  is  no  such 
sketch-based  database  that  includes  this  information  at  this  time.  Therefore, 
605  we  plan  on  making  one  publicly  available. 

Additionally,  we  hope  combine  the  approach  used  in  this  paper  with  param¬ 
eter  estimation  and  feature  selection.  The  addition  of  parameter  estimation  will 
hopefully  provide  improved  results  by  having  a  variable  smoothing  parameter 
opposed  to  static  one.  Feature  selection  methods,  such  as  KPCA,  PCA,  or  ICA, 
6io  provide  the  means  to  utilize  the  most  meaningful  features,  which  will  provide  a 
more  compact  and  useful  descriptor  for  a  sketch-based  password. 
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